Using Electronic Health Records to Enhance Lyme Disease Surveillance: Protocol for the SubLyme Network

Introduction

Lyme disease is the most common vector-borne illness in the United States, occurring in 2 major foci: the Northeast and mid-Atlantic regions and the upper Midwest [1]. Its incidence and geographic range are increasing, posing a significant public health burden [1]. Lyme disease surveillance data have been limited by the clinical and diagnostic complexities of the disease and the burden of disease reporting on clinicians and health departments [2]. A more robust disease surveillance strategy is essential to inform prevention, detection, and treatment strategies for populations and communities at greatest risk [2].

Lyme disease has been a nationally notifiable condition in the United States since 1991. Clinicians and laboratories report potential cases to state or local health officials, who classify these reports according to standardized surveillance case definitions developed by the Council of State and Territorial Epidemiologists (CSTE) [2]. Until recently, the CSTE case definitions required clinical details to determine case status, regardless of geography or frequency of the disease in the jurisdiction [2]. As case counts escalated in high-incidence regions, the workload of reviewing and validating clinical findings exceeded available resources, resulting in underestimates of Lyme disease incidence and variability in the quality and content of reported data across jurisdictions [2].

To address these limitations in systematic surveillance, the most recent CSTE case definition, updated in 2022, only requires a positive laboratory test in high-incidence states, with or without clinical confirmation, to classify someone as a probable case [3]. While it has likely improved consistency in reporting and reduced reporting burden, this update to the case definition could still be challenged by both overreporting and underreporting of disease. Using a definition based on serological testing alone might result in the inclusion of clinically incompatible or nonincident cases (ie, positive laboratory test result due to a previous infection) [2]. Conversely, it is probable that the new surveillance criteria do not capture cases for which laboratory evidence is not available, either because of insufficient time to build a detectable antibody response or because cases are presumptively treated without serological testing [2].

Electronic health record (EHR) data have the potential to overcome the challenges of traditional approaches to surveillance. Health system networks have been working together to develop EHR-based surveillance for chronic [4,5] and infectious diseases [6], with the goal of enriching traditional surveillance data with more timely and efficient approaches. EHR systems are widespread across the United States, with more than 90% of hospitals and nearly 90% of office-based physicians using an EHR system [7]. Clinical and demographic data are routinely collected and stored in EHR systems for clinical care purposes and can be used to produce timely surveillance estimates, without additional data collection or reporting. In EHR-based surveillance, case ascertainment is generally based on an automated EHR-based algorithm (ie, computable phenotype [CP]) rather than manual reporting and case classification methods. CPs are implemented using programming that can be quickly modified as needed, making this approach flexible compared to the more complex process of altering traditional surveillance strategies (eg, training of clinicians and health department personnel, revising data collection protocols, and undergoing consensus processes to changing CSTE case definitions) [4].

EHR systems have the potential to address many of the reasons for underreporting while reducing the labor needed for surveillance. Furthermore, the large population sizes captured by EHR systems have the potential to overcome sample size challenges of identifying and studying the less common, but sometimes severe, manifestations of disseminated Lyme disease (eg, carditis) [1]. Additionally, the longitudinal data in the EHR can be used to better understand the burden of posttreatment Lyme disease syndrome, and the large patient populations can be used to study outcomes in special populations (eg, pregnant women) and among demographic subgroups [8].

In 2022, the Centers for Disease Control and Prevention (CDC) issued a broad agency announcement requesting proposals to strengthen Lyme disease surveillance and research using EHR data. Awardees worked with the CDC to establish the Surveillance Based Lyme Disease Network (SubLyme). SubLyme will conduct network-wide Lyme disease surveillance in endemic regions, while informing future EHR-based surveillance strategies for Lyme disease at a national level. This paper describes SubLyme, its structure, and the methods that will be used to achieve the network objectives.

Methods

Study Design

SubLyme has three primary objectives: (1) to establish and evaluate criteria for identifying Lyme disease cases in EHR data (ie, create CPs) that can be scaled across diverse health systems, (2) to use the CPs to estimate incidence, and (3) to describe Lyme disease incidence by case demographics. A secondary objective is to develop alternative CPs that can distinguish between acute and disseminated Lyme disease and identify specific clinical manifestations (eg, dermatologic, neurological, joint, and cardiac) for routine estimates of disease severity. This systematic collection of standardized, real-time data on cases will enable researchers to monitor trends, identify risk factors, and evaluate the impact of prevention and treatment strategies. The protocol for achieving the primary aims is described in detail below.

Ethical Considerations

Each SubLyme site, the coordinating center, and CDC received approval and waivers of consent from their local institutional review boards for this protocol: CDC (nonhuman subjects research determination), Geisinger (2016-0365), MaineHealth (2155117-7), Marshfield Clinic Research Institute (IRB-24-1487), Mass General Brigham (2023P002994), Tufts Health Sciences (STUDY00004649), and Westat (nonhuman subjects research determination).

Study Setting and Population

SubLyme includes EHR data from patients in 5 health systems in 3 US regions highly endemic to Lyme disease: Geisinger (Pennsylvania); Marshfield Clinic Health System (Wisconsin); and 3 health systems in New England—Mass General Brigham, Tufts Medical Center, and MaineHealth; the network is administered by a coordinating center (Westat) and directed by CDC [9]. The network covers 38 counties in Central and Northeastern Pennsylvania, 5 counties in Southern New Hampshire, 1 county each in Rhode Island and Connecticut, 70 counties in Wisconsin, and all counties in Maine and Massachusetts (Figure 1).

‎

Data Sources

EHR data from each of the 5 participating health systems will be the source of SubLyme data. Data elements will be extracted in the categories of demographics (eg, age, sex, race, ethnicity, and residential address), medication orders and medication indications, diagnoses, laboratory orders and results, clinical encounters (eg, outpatient, inpatient, and emergency department), and corresponding dates.

Development of CPs

We are evaluating the Lyme disease CPs based on previously published methods of CP validation [10,11]. Each SubLyme site identified a Lyme disease wide-net cohort (those potentially suspected of having Lyme disease), defined as patients with any Lyme disease element in their EHR (International Classification of Diseases-10 diagnosis codes for Lyme disease, Lyme disease laboratory test orders using Current Procedural Terminology codes, and Lyme disease–appropriate antibiotic orders) during 2022 to 2023 (Multimedia Appendix 1). This approach was designed to ensure high sensitivity. Thus, individuals not identified by these criteria were assumed not to have had Lyme disease in these years (ie, true negatives [TNs]). Then, patients in the wide net were categorized into 1 of 7 categories based on presence of different combinations of these Lyme disease elements within the calendar year (Figure 2). Next, sites created a reference panel for CP validation by randomly sampling 500 patients from each site in the wide-net population (2500 patients total), stratified by the 7 Lyme disease element categories.

‎

Chart Review

Sites conducted manual chart reviews of the reference panel sample, classifying patients into 1 of 5 categories to approximate CSTE 2017 criteria: likely Lyme disease (clinician observed rash and documented “erythema migrans” or “EM” or clinician observed or diagnosed Lyme arthritis, neuroborreliosis, or carditis and a positive laboratory test), possible Lyme disease (rash consistent with erythema migrans or nonspecific Lyme disease signs or symptoms with a positive, 2-tiered laboratory result for Lyme disease), not incident Lyme disease (evidence that patient presented with persistent symptoms from an earlier-occurring Lyme disease or the patient had a positive, 2-tiered laboratory result for Lyme disease in the absence of signs or symptoms), not Lyme disease, and insufficient information.

The chart review process was harmonized across SubLyme sites, facilitated by a standardized REDCap (version 17.1.2, Vanderbilt University) [12] abstraction form. Detailed instructions for chart reviewers were provided in a manual of procedures. Chart reviewers were trained during SubLyme meetings and internally at each site. Each site implemented interrater reliability assessments at the start of the study to calibrate chart abstraction and classification of case status. These efforts included independent review of selected records by multiple reviewers, reconciliation of discrepancies, and iterative refinement of the manual of procedures. The size and composition of chart review teams varied by site, based on staff availability. In the case of disagreements that could not be reconciled, teams consulted a physician for the final decision. Sites also brought challenging chart review cases to network-wide meetings for discussion and to reinforce consistent coding practices. Reviewers were not blinded to the CP status while conducting the chart review.

Statistical Analysis

The network will evaluate the validity of EHR-based CP definitions, including both a priori, rule-based CPs (based on algorithms defined by subject matter experts) and machine learning–based CPs. For all CPs, sensitivity will be calculated as the true positives (TPs; classified as having Lyme disease by chart review and by CP) divided by all those classified as having Lyme disease by chart review (TP/[TP+false negatives]). Specificity will be calculated as the TNs (classified as not having Lyme disease by chart review and by CP) divided by all those classified as not having Lyme disease by chart review (TN/[TN+false positives]). Positive predictive value (PPV) will be calculated as the TPs divided by all those classified as having Lyme disease by CP (TP/[TP+false positives]). Finally, negative predictive value will be calculated as the TNs divided by all those classified as not having Lyme disease by CP (TN/[TN+false negatives]). We will create these metrics for 2 different definitions of TP, one that classifies only likely Lyme disease cases as positive for Lyme disease and one that classifies both likely and possible Lyme disease cases as positive for Lyme disease.

Measures will be primarily calculated combining all sites, with site-specific measures calculated as supplemental analysis. To account for stratified selection into the reference panel sample by Lyme disease element category, weighted measures will be calculated such that for each site, the proportion of total weights for observations in each category is equal to the proportion of wide-net events in that category. Final selection of the CPs for generating incidence estimates will be based on CP performance as well as other CDC standards for surveillance, including simplicity and timeliness [13].

Machine Learning Analysis

Machine learning–based CPs will be developed using recursive partitioning trees and extensions thereof, including random forests and gradient boosted trees. Model predictors will be derived from structured EHR elements, including diagnosis codes, medication orders, and laboratory testing. Data will be split into separate training and testing subsets, with model performance assessed in the testing set.

Incidence Estimation

Incidence calculations require estimation of the population at risk (“denominator”). Consistent with other EHR-based surveillance networks [4], each SubLyme site defined the denominator for incidence estimates using residential address and health care use history to identify the population from which Lyme disease status will likely be recorded in the health system’s EHR. Denominator approaches vary by SubLyme site based on health system–specific factors (eg, market share and geographic service area; Table 1). The numerator for incidence will be patients from the denominator populations who meet criteria for validated CPs. Each SubLyme site will provide unadjusted incidence rates overall and for demographic and geographic subgroups. The sensitivity and specificity of the CPs will be used to adjust for misclassification of cases and noncases through a Bayesian model. Specifically, the number of observed events meeting the CP will be modeled as a binomial distribution, with the probability of a positive classification defined as sensitivity×p_{True Case}+specificity×(1−p_TrueCase) The number of trials will equal the number of wide-net events, and prior distributions for sensitivity and specificity will be specified based on the values estimated during CP validation [14]. The 50th, 2.5th, and 97.5th percentiles of the resulting posterior distribution will be taken as the point estimate and 95% credible interval of the adjusted incidence numerator and adjusted incidence, calculated using the denominators described above.

Missing Data

Missingness in EHR-based surveillance may be due to data quality issues in the extraction and transmission of data or could reflect data elements that are truly missing from a health system’s EHR due to clinical practices or documentation workflows [15]. We will assess data quality by evaluating the stability of missingness across time and consistency of missingness across sites. The coordinating center will work with sites to resolve missing data issues due to errors in data extraction or transmission. Consistent with other EHR-based surveillance networks, if it is determined that missingness correctly reflects the data from the EHR, no additional actions will be taken for incidence estimation [16]. As SubLyme moves into inferential analyses, more formal approaches to missingness will be considered, such as multiple imputation or inverse probability weighting.

Data Sharing and Management

To facilitate network-wide analyses at the coordinating center, each site executed a data use agreement that permits the sharing of a limited dataset that includes chart review data and EHR data elements (eg, demographic characteristics, diagnoses, medication orders, laboratory orders, laboratory results, and relevant dates masked via date shifting) with the coordinating center. The coordinating center retains the data shared from each site. Only the site and the coordinating center have access to line-level data. Sites do not have access to line-level data from other sites. Data transfers between sites and the coordinating center are conducted via a secure file transfer protocol. The coordinating center manages these data centrally on a secure central platform with technical safeguards that include unique user identification and automatic log off. Sites have access to their own individual-level data and aggregate data from the network.

Results

In preparation for the CP validation, each SubLyme site identified their wide-net population (Table 2). In total, the SubLyme wide-net population had 332,256 patients identified from more than 4.6 million patients. The wide-net cohort comprised 55.6% (n=184,734) female patients, 87.9% (n=292,053) White patients, and 90.8% (n=301,688) non-Hispanic patients. In total, 9.6% (n=31,896) of patients were aged 0 to 19 years, 20.2% (n=67,115) were aged 20 to 39 years, 46.2% (n=153,502) were aged 40 to 69 years, and 24% (n=79,741) were aged ≥70 years.

Among the wide-net cohort, more than half (n=177,092, 53.3%) only had a Lyme disease–appropriate medication order and 35.8% (n=118,948) only had a laboratory order for a Lyme disease test (Table 3). Among individuals with at least 2 Lyme disease elements, the most common was the combination of a Lyme disease–appropriate medication order with an order for a Lyme disease test (n=22,926, 6.9%), followed by the combination of a Lyme disease diagnosis, Lyme disease test order, and a corresponding medication order (n=5316, 1.6%). The least common combinations were those that did not include an order for a Lyme disease–appropriate medication.

Discussion

The exploration of novel surveillance methods for Lyme disease is increasingly important as disease incidence rises and the geographic range of the disease expands [1]. An EHR-based approach to surveillance has the potential to overcome existing barriers to case reporting and limitations of case criteria that impact the quality of current surveillance strategies and offer new opportunities for research. As a collaboration between CDC, large health care networks in endemic regions, and a central coordinating group, SubLyme is well positioned to advance Lyme disease surveillance and research through the use of EHR data. SubLyme will generate incidence estimates for regions of the United States with some of the highest Lyme disease burdens and will inform future strategies for enhancing nationwide surveillance using EHR data.

Targeted deployment of effective prevention, detection, and treatment strategies for disease requires high-quality surveillance data. There are emerging opportunities for prevention, including multiple Lyme disease vaccines at various stages of development, with clinical trials currently underway [2,17]. Enhanced surveillance will be essential for understanding Lyme disease vaccine uptake and for informing the design of studies to monitor vaccine safety and effectiveness. Over the last decade, EHR data have been increasingly used to monitor the safety and effectiveness of vaccines (eg, COVID-19, human papillomavirus, influenza, and pneumonia) [18,19] and will likely be critical in optimizing the impact of future Lyme disease vaccines and other novel preventive interventions.

Prior research demonstrates the feasibility of using EHR data to study Lyme disease [8,20-23]. SubLyme investigators have used EHR data to develop Lyme disease CPs [20,21] and to study individual and community-level risk factors for Lyme disease and Lyme disease stage and manifestation [8,23]. These studies have consistently reported higher estimates of Lyme disease incidence than more traditional surveillance approaches [8,23], in part because EHR-based estimates are not dependent on the time and effort of clinicians and health departments to report cases. By contrast, the EHR-based CPs used for generating these estimates also had moderate PPV and variable specificity, potentially resulting in inflated incidence estimates [20,21]. Nevertheless, EHR-based studies have resulted in generally similar demographic and seasonal patterns as those reported through traditional surveillance approaches over decades, lending confidence to these approaches [8,20].

EHR-based surveillance efforts face challenges. First, as noted above, Lyme disease CPs have been shown to have moderate performance in terms of PPV and specificity [20,21]. SubLyme will explore machine learning approaches to CPs and refine rule-based CP components (eg, laboratory results and antibiotics) to improve CP performance and the quality of incidence estimates. Second, EHR data are limited to individuals affiliated with the reporting health systems, and these populations may differ from the general population [24]. EHRs include data on the subset of the population that seeks care, potentially biasing EHRs toward greater coverage of women, children, and individuals who are more ill [4,24]. Third, the patient populations served by the SubLyme health systems are predominately White and non-Hispanic, potentially limiting generalizability to endemic regions of the country with different demographic profiles. However, the network includes thousands of individuals in different racial and ethnic groups, allowing for reporting of Lyme disease incidence in these subgroups. Additionally, CPs may perform differently in emerging or low-incidence states. Finally, manual review of EHR data is an imperfect gold standard for determining true Lyme disease case status due to the limitations of Lyme disease testing early in disease, inaccuracy of some diagnostic coding practices, and variability in clinician documentation.

SubLyme has important strengths. The health systems in the network are located across diverse geographic regions of the United States (Upper Midwest, Northeast, and Mid-Atlantic) and serve a range of community types, including urban and rural populations. Clinical manifestations of Lyme disease have been found to differ in regions of the Northeast and Midwest, attributed, in part, to differences in the genetic structure of the bacteria that cause Lyme disease in these regions [25,26]. Prior work has shown differences in Lyme disease risk and risk factors by degree of urbanicity [22]. The systems also differ in terms of their data access (eg, access to historic EHR data and diagnostic coding practices) and analytic infrastructure (eg, established data sharing practices and dedicated data analysts) that support EHR-based surveillance. Thus, the approach to surveillance will be designed to be flexible and generalizable to a broad range of health systems. SubLyme is multidisciplinary, with experts in epidemiology, infectious disease, vector-borne disease, molecular biology, disease surveillance, Lyme disease, computational biology, and EHR-based epidemiology, with diverse perspectives informing well-rounded surveillance strategies. Finally, in addition to advancing surveillance methodology, SubLyme will identify large cohorts of well-characterized individuals with Lyme disease for whom longitudinal EHR data are available across the network. The data available on these cohorts can inform future research in a variety of areas, including Lyme disease staging and manifestations; posttreatment Lyme disease syndrome; and, as previously noted, vaccine safety and effectiveness.